A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment

A central problem in stereo matching by computing correlation or sum of squared differences (SSD) lies in selecting an appropriate window size. If the window is too small and does not cover enough intensity variation, it gives a poor disparity estimate, because the signal (intensity variation) to noise ratio is low. If, on the other hand, the window is too large and covers a region in which the depth of scene points varies, then the disparity within the window is not constant. As a result, the position of maximum correlation or minimum SSD may not represent a correct estimate of disparity. For this reason, an appropriate window size must be selected locally. There has been, however, little research directed toward the adaptive selection of matching windows. The stereo algorithm we propose in this paper selects a window adaptively by evaluating the local variation of the intensity and the disparity. We employ a statistical model that represents uncertainty of disparity of points over the window: the uncertainty is assumed to increase with the distance of the point from the center point. This modeling enables us to assess how disparity variation within a window affects the estimation of disparity. As a result, we can compute the uncertainty of the disparity estimate which takes into account both intensity and disparity variances. So, the algorithm can search for a window that produces the estimate of disparity with the least uncertainty for each pixel of an image. The method controls not only the size but also the shape (rectangle) of the window. The algorithm has been tested on both synthetic and real images, and the quality of the disparity maps obtained demonstrates the effectiveness of the algorithm.